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ABSTRACT 

The United Nations Educational, Scientific, and 
Cultural Organization (UNESCO) held a meeting of experts on education 
indicators to discuss the role of these indicators in the 1991 World 
Education Report as well as future issues concerning education 
indicators. The meeting was opened by the chairperson with a review 
of the usefulness of indicators in providing summary measures of the 
state of education in the world and in individual nations. The group 
discussed desirable traits of education indicators. The need for the 
measurement of actual learning, rather than simple attendance was 
highlighted as well as the need to measure teachers' contributions 
and roles in the report. The discussion then focused on the value of 
individual indicators and UNESCO's overall measurement strategy. A 
general debate included the following areas! (1) coverage; (2) 
framework; (3) presentation; (4) number of indicators used; (5) 
disparities; (6) vulnerable groups; (7) illiteracy; (8) issues 
related to language and script; (9) learning outcomes and efficiency; 
(10) projections; (11) book and textbook production; (12) enrollment 
ratios; (13) teacher supply; (14) transition rates; and (15) meaning 
of public/private distinction. Presentations followed on educational 
indicator work done by the Organization for Economic Co-operation and 
Development (OECD) and the International Association for the 
Evaluation of Educational Achievement (IEA) . The meeting concluded 
with a discussion on future UNESCO work on education indicators and a 
suggestion for a similar forum be held in regard to the 1993 World 
Education Report. (CK) 
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Informal Meeting of Experts on Education Indicators 
PariSt 31 January - 5 February 1991 



REPORT 



The Meeting was opened by Mr Carceles, Director, Division of 

Statistics, who chaired the meeting. Mr Power, Assistant 

Director-General for Education and Mr Smyth, Co-ordinator of the World 
Education Report, 1991 also participated* 



Mr Carceles opened the meeting by telling the group what UNESCO's 
general aspirations were in the field of educational L Jicators and 
UNESCO's expectations as regards this meeting. He said they were 
looking for advice from the group about the role of such indicators 
both in the 1991 World Education Report currently being produced and 
in future issues* They were also looking for advice on the long-term 
prospects of further development of the indicators for use in a 
variety of situations expecially in relation to priority programmes of 
UNESCO* It was hoped that the indicators would also be uceful to 
persons concerned with planning and investment in education* They 
would be part of a wider data-base available in UNESCO* He hoped that 
the indicators developed would provide a guide to member countries 
particularly developing countries in creating their own indicators to 
help locate their problems in a comparative setting in order to 
monitor their progress towards solving these problems. 

Mr Power said that UNESCO looked to an indicator system to tell "what 
was going on in education" in summary form and to provide a guide to 
the policy issues which both int rnational agencies and individual 
countries faced* The indicators should be "doable", affordable and 
relate to the real situation. In so far as was feasible within cost 
limits, they should provide information about the quality of learning 
as veil as about participants in activities designed to foster 
learning and should if possible have a diagnostic role* The 
participants should analyse the strengths and weaknesses of the 
current approach to educational indicators, should identify the 
priorities for future development and could usefully consider the 
relationship of these indicators to those in other fields* He pointed 
out that indicators were necessarily used in political debate both at 
national and international level and said that the indicators should 
he robust enough to be able to survive the criticism which such debate 
usually engendered* Finally he referred to the work that OECD had 
done in this area and said that UNESCO looked forward to benefitting 
from this work. 

Indicators for the World Education Report 

Mr Smyth then introduced the UNESCO approach to the 1991 World 
education Report* He identified some of the key concerns of the 
report such as a newly intensified preoccupation with the extent to 
which children actually learned some of the things which educators 
felt they ought to learn, as distinct from simply participating in 
organised schooling or other teaching/lemming situations* Be also 
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said thnt the report would emphasize the critical role of the teacher, 
who was not simply another resource input, one more expensive and less 
easily replaced where necessary than other resources. He asked 
whether the revised stress on the importance of the classroom teacher 
was adequately reflected in the current indicators and suggested that 
they would need modifications in this area. He drew attention to the 
variety of audiences which they hoped to reach and the need to cater 
*• f*r *» possible to the special needs of these audiences. 

Mr Hyland introduced the discussion paper and drew attention to the 
fact that two macro-issues need to be addressed a substantive one as 
to what indicators should be modified or dropped and what new 
indicators should be added and a more general one of evaluating the 
current UNESCO approach and proposing and evaluating alternative 
approaches either as complementary or competitive to the present 
approach. He pointed out that any alternative approach had cost 
implications. 

The meeting was then opened for debate on general issues. The 
following topics were discussed. 

Coverage: A number of participants expressed concern that the 
indicators did not cover what they referred to as non-formal 
education. They expressed the view that this was an important and 
perhaps growing aspect of education and gave a number of examples in 
many countries. They raised the question of how it might be covered 
in the future. The Secretariat pointed out that efforts to collect 
data on these activities had been made and had not been successful due 
to the non-availaui 1 ity of data at the national level. The great 
variety of programmes covered by the term non -formal also created 
problems. However, it was agreed that the issue must be reopened. 

Framework: A number of participants stressed the need for a 
satisfactory theoretical framework as a basis for the identification 
of gaps in the indicator set and of discussing priorities for 
improvement. The form which such a framework should preferably take 
(e.g. whether formal models of an education system should be used or 
not) and the need for UNESCO to avoid anything which might freeze 
thinking about a vast variety of unpredictable problems and 
peremptions about these problems were discussed. 

Mr Nuttall while stressing the need for a framework informed the 
meeting tht the original 4-way framework (i.e. Environment, 
Resources, Process and Output) adopted by the OECD as an organizing 
- schema was under reconsideration and was unlikely to survive in that 

form? an agreed alternative was not yet available. Mr Wiley suggested 
the use of a system of categories relating to essential functions of 
any organized educatioa/schooling system; he referred to five 
headings: Pre-conditions of a schooling system.; Resources required, 
distinguishing those capable of being designated in terms of a 
currency aad others of a different type such as parental or 
administrative attitudes; Service Provision, distinguishing 
organizational aspects linked to Resource allocation from the delivery 
of services to clients; Participation by clients identifying such 
items as time spent as well as more traditional items and Outcomes, 
distinguishing learning achievements from ivediate and long-term 
destination, including in this item participation not only in the 
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labour force but in other key aspects of society. He pointed out that 
a system of this type could set an agenda for development. Mr Smyth 
pointed out that an implicit set of categories for organizing and 
evaluating the adequacy of the set of indicators was provided by the 
structure of the World Education Report and the issues highlighted in 
it. Apart from the question of a formal framework, participants 
discussed alternative criteria for creating and rejecting indicators. 
One proposal was that three basic determinants should be used (a) 
policy relevance, (b) compatibility with an agreed conceptual 
framework and (c) feasibility, it being recognized that feasibility 
had many elements, especially quality; cost; time. Another 
participant proposed a different way of organizing indicators. He 
believed they could be classified by (a) role in monitoring policy and 
identifying the need for changes; (b) guidelines as to the direction 
in which changes needed to be ( made and (c) help in monitoring the 
consequences of reforms. 

P resentation : Many participants jit that the impact of the World 
Education Report and related sets of indicators would be greatly 
enhanced if economic and demographic indicators from other UN 
Organisations were included to provide summary data on the wider 
situation in each country. The feeling of the meeting was that the 
gain from reproducing other indicators in conjunction with educational 
indicators was likely to be so great that the natter should be 
considered by the Secretariat* 

Number of indicator s used ; The above issue and the Secretariat 
reaction to some other specific suggestions, led to a general 
discussion of the appropriate target size for the indicator section of 
the report and the problem of trade-offs. It was felt that the 
criterion of size tended to avoid facing the issues of priorities in a 
formal way and had underlying conservative implications, in that it 
led to the repeated inclusion of indicators shown in previous years, 
at the expense of new options* 

Disparities ; There was a widespread feeling at the meeting that it 
would be desirable to show disparities within countries as well as 
averages; a number of preliminary proposals were made. It was however 
widely recognized that a deeper consideration of the possibilities was 
required and that the item would be taken up again under Item 4 of the 
agenda. 

Vulnerable groups ; Apart from the general issue of disparities, many 
participants felt that indicators illustrating the situation of 
vulnerable or disadvantaged groups, including women 'in some countries, 
within the more general society would be highly .desirable. Several 
sugges t i ons we r e made • 

Illiteracy ; Compared with other disadvantaged persons and groups, the 
problems of persons who were functionally illiterate (and probably 
innumerate as well) had a special call on the attention of UHESCO. 
The feeling was that while current indicators in this field were 
useful it would be desirable to expand the coverage of this item. 
Many participants felt that rates of illiteracy should be shown for 
specific age groups of particular interest. One participant suggested 
that cohort analysis would help to illustrate the dynamics of the 
situation as well as showing the effects (if any) of particular 
programmes. 
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Issues related to Language and Script : A number of participants 
pointed out that the variety of languages, and in some cases scripts, 
used within particular countries posed a great challenge for already 
hard-pressed educational authorities. Such challenges were not always 
adequately set. It would be desirable to have indicators illustrating 
the challenge and where available the extent and adequacy or otherwise 
of the response. The Secretariat rointed out that data on some 
aspect* of this issue were beir^ collected in the special one-off 
questionnaire recently issued on Basic Education. Data would not be 
available for the 1991 World Education Report but might well be 
available for the 1993 World Education Report. 

Learning Outcomes and Efficiency ; A number of participants raised a 
variety of problems in these fields. It was recognised that the wider 
issue of measuring the extent and patterns of actual learning outcomes 
was a formidable task and would need to be discussed again following 
the presentation of the International Association for the Evaluation 
of Educational Achievement ( IEA) . However, attendance at school, as 
distinct from enrolment, was one vital aspect of this problem and some 
participants felt that an indicator to highlight this issue was 
essential. The Secretariat pointed out that it appeared that very few 
countries had data on this matter but that they would reinvestigate 
the issue. A number of possibilities for further indicators to show 
various aspects of the efficiency of school systems were mentioned but 
it was not evident that a better suggestion than the existing 
indicator was available. 

Projections : It was agreed that it was not desirable to use the 
formal indicator structure to attempt to summarize the results of 
projection studies carried out by UNESCO. It was agreed also that it 
would be essential to draw attention to the salient aspects of such 
studies in the report, perhaps confining identification of particular 
countries to acute situations. 

Book and Textbook Production ; An extensive discussion took place on 
these issues. There were at least three interrelated issues and data 
on Production of new titles covered only a limited part of the 
question. The number of actual books produced, as well as import and 
export of books by language/cultural categories was also relevant. As 
well as data on books generally it would be highly desirable to have 
data on text-books, preferably by level and by subject, at secondary 
and higher levels. The position as regards school libraries was also 
relevant. The Secretariat pointed out that attempts to collect such 
data had been made with very little success. Some participants felt 
that it was as important to highlight the absence of data in these key 
fields as it was to present one or two items of the very limited data 
available. 

Enrolment ratios ; The possibility of using age-specific enrolment 
ratios (ASER) in various ways, to show ASER at critical ages; to 
identify the modal age and measures of dispersion both at the expected 
intake age and for primary (or basic) education generally, was 
canvassed. One participant expressed a Willingness to look at the 
data available to UNESCO and make further specific suggestions. 
Support for the suggestion made at para. 4a of the discussion paper 
was expressed. 
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Sundry specific points : A number of important specific points were 
made in addition to the above, some methodological and soma 
substantive* Among the former were suggestons as to the use of 
housenold surveys, the use of micro-data and of population censuses as 
ways cf collecting educationally relevant data capable of being 
presented in indicator form, particularly on such topics as activity 
in the "non-formal" area, which could not be got through 
institutional-based questionnaires* In another intervention the 
question of how quantitative judgements, recognized as playing an 
important role in educational decision-raaking could be reflected in 
indicator presentation was raised* This was related to the concern 
expressed by some participants that the indicator system should if 
possible not only identify critical problems but should illustrate why 
such problems occurred* 

The possibility of presenting indicators created by using number of 
institutions or schools was canvassed* The Secretariat acknowledged 
that such data were collected and that such indicators would be 
desirable but were fearful that the quality of the data provided would 
not sustain indicators of adequate quality* This led to a general 
discussion of the problem of using minimum standards of quality and 
reliability to eliminate particular indicators, to censor or footnote 
particular entries; to provide information of a contextual kind to 
help sophisticated interpreters and to prevent naiv^ 
misinterpretations* Several instances of probable incompat iblity were 
pointed out* The Secretariat expressed willingness to consider these 
issues * 

The Secretariat promised to consider all of these suggestions 
constructively and sympathetically, insofar as available data and 
resources allowed* Extensive and detailed discussions then took place 
on each of the Tables I to VI, including the possibility of expanding 
and reorganizing the Tables to the limits allowed by the Co-ordinator 
of the World Education Report, and in other contexts* 

These discussions are not covered in detail in this presentation apart 
from some specific items mentioned below, as the Secretariat indicated 
that they interpreted the feeling of the meeting to accord priority to 
some suggested changes over other suggestions* They promised to pay 
particular attention to these priority suggestions, (a) in modifying 
what will be presented in the 1991 World Education Report subject to 
time limitations and (b) in a wider and less constrained context of 
the 1993 and following editions of the World Education Report, and in 
other contexts* 

Special Issues 

Teacher supply : It was regarded as not feasible to illustrate the full 
dynamics of teacher supply in the indicator system for obvious 
reasons, but some key indicators could usefully be presented. One, to 
illustrate the relative constraints on the availability of teachers of 
good quality might consist of showing the ratio of number of teachers 
to one or several aspects of the adult population or the labour force. 
The following possibilities were canvassed; the literate adult 
population, the non-agricultural labour force, the total labour force. 
It was also felt to be desirable to identify the number of third level 
students classified as in the field of "Education", as a focus 
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indicator particularly relevant to the World Education Report* The 
possibilities of misleading readers because of various 
incoaparabilities in structures leading to becoming a teacher as well 
as other incompatibilities based on changing conditions as regards 
the supply and demand for teachers were mentioned. On balance it 
seemed worthwhile to provide this indicator. 

Transition rates : Participants enquired why transition rates from 
completed primary to secondary and from completed secondary to third 
level were not included in the indicators. The Secretariat indicated 
that the former, if not the latter, were always considered each ti*ae 
the revision of the set of indicators was discussed, but that 
difficulties in the data provided by a number of important countries 
inhibited the Secretariat from including these indicators. Following 
an extensive discussion the feeling of the meeting was that an 
indicator expressing the transition from primary to secondary should 
be provided even if data for certain important countries was not 
suitable to allow them to be included. 

Meaning of public/private distinction : A number of participants drew 
attention to ambiguities in the interpretation of this distinction. 
In particular it was not clear Y • schools which were legally private, 
but which were financed predora' tly from public funds (including the 
payment of all or part of teacher salaries) were expected to be 
handled by individual countries. It was recognized that very little 
could be done to clarify the situation for the 1991 World Education 
Report, but that further work to clarify the situation both as regards 
the creation of a sensible paradigm to handle the issue and to 
discover country practice should be done. 

Modification of stand a rd questionnaire to identify more accurately the 
position as regards compulsory attendance 

It was pointed out that the question on compulsory attendance in the 
standard questionnaire was not adequate to identify the position in 
countries or component parts of member states which did not define 
their regulations in terms of, or exclusively in terms of, age. In 
addition, those cases where compulsory part-time enrolment in schools 
(sometimes as part of a "co-op" or a "dual" system) was required 
following the completion of full-tiue compulsory enrolment was also 
not covered adequately. Finally some participants pointed out that 
the implementation of these (regulations differed significantly from 
country to country, and perhaps within countries, and that data on the 
degree of implementation would be essential to understand the actual 
role that compulsion as such played in each country. The Secretariat 
acknowledged that the questionnaire should be reviewed on this issue. 

O verview of OBCD work in the area of education indicators 

The Chairman of the Meeting, 0. Nascimento, Chief, Section of 
Statistics on Education, Division of Statistics, gave the floor to Mr 
Nuttall who is a meaber of the Co-ordinating Group (COG) tot the OBCD 
Project and who provided information on the OBCD activities. In his 
presentation he distinguished between organizational matters, expected 
scientific outcomes and the expected products of theme activities at 
the end of the second phase, now reaching a climax. Hr Nuttall 
identified five problems he sees emerging from theme activities! the 
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implications for the work UNESCO had already undertaken on the 
revision of ISCED; the need to maintain cohesion between the work of 
the various groups and networks to whoa responsibility for particular 
areas had been devolved; the time and money required for the 
enterprise and the need to maintain the interest of sponsors over an 
extended period; the special problems concerned with the measurement 
of student outcomes and the way of handling variations in country 
interpretations of the templates and other key documents. 

A wide range of questions, most of them of a clarif icatory nature, 
followed. In reply Mr Nuttall provided additional insights on the 
OECD activities as they unfolded over time* The Chairman raised three 
questions. How might the OECD work be used by UNESCO in its ongoing 
concern with improving the situation including information to and 
about developing countries; how these activities were seen as 
interfacing with the regular joint collection of educational 
statistics and in at ion to the work already initiated on the 
revision of ISCL what the best basis for co-operation with OECD 
might be. Mr Nuttall offered some thoughts on these topics and said 
that he felt sure that OECD valued and wished to continue the ongoing 
co-operation in data collection with UNESCO and EUROST AT • He referred 
to the opportunity the project gave for learning from country 
experience and the stimulus it gave the countries to resolve festering 
issues in the area of international comparability. He pointed out 
that at present the statistical data needed to calculate the 
indicators were collected from countries and the calculation of the 
indicators as such was done by OECD. He expressed the view that many 
even well informed people tended to expect too much from an indicator 
system. In his view a good indicator system would direct attention to 
problem areas or to areas with special potential. It would then be 
necessary for the concerned authorities to collect additional 
information before deciding on necessary action* what was necessary 
now in the contet . of the OECD project was to decide what could 
usefully be done in the short term. Mr Smyth raised the question 
whether any way could be found of expediting the revision of ISCED. A 
number of participants raised the question how the IUES project 
intended to handle the issue cf non-formal and informal educational 
and training activities. Mr Nuttall informed them that the OECD 
project was primarily concerned with the formal system, at least 
during this phase of the work, though it did in principle cover adult 
education and related activities also. 

A general discussion followed of the role of non-formal and informal 
education in developing countries, and of ways of clarifying the 
ambiguities to which the term "non-f ormal" in particular gives rise. 
One participant suggested that it would help resolve some of these 
difficulties if we could list the various types of activities covered 
by these terms and subsequently develop a classification system which 
would enable us to refer to them without ambiguUy; identify the 
salient aspects of each sub-system and in due course collect data on 
them in a more systematic manner. We need to know for each sub-system 
what the variants of service provision *re and what the term 
participant Mans in context. Identification and classification of 
institutional mechanisms that deliver services and allocate resources 
could also make a useful contribution. Another participant stressed 
thm extremely wide range of activities covered by the term and saw it 
am essentially a basket termi everything dealing with designed 
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learning not allocated to the regular school and college system fell 
into it* He felt that we already knew enough from previous studies 
and reports to use existing classification patterns. Other 
participants disagreed and expressed the view that no cohesive 
category system existed which allowed activities to be placed 
unambiguously and that a suitable structure was necessary to underpin 
systematic data collection and analysis. 

Another participant suggested that a functional structure using goal 
and content as a basis was a necessary aspect of any viable 
classification system. One of the problems of overall classification 
of educational activities was that the same activity might be regarded 
as non-formal education in one country and as part of the regular 
system in another. The Secretariat referred to surveys done in some 
countries and said that while they collected a great deal of 
information it was hard to identify systematic patterns or to present 
the data in meaningful summary fashion due to the diversity of the 
terminology used and of the basic structures encountered. The point 
was also made that some of the relevant activities were multipurpose 
and could not easily be classified by goal. The suggestion was made 
that regional offices might be asked to collect some data on 
non-formal education using existing questionnaires as a pragmatic way 
of increasing information about the activities concerned. The 
question of what individual countries had done and what they were 
regularly publishing about these activities was also raised. 

Mr Plank, at the invitation of the Chairman, outlined the developments 
in one of the OriD key group "Costs and Expenditure". He indicated 
that the group initially covered costs and resources but that the item 
"resources" was hived off and allocated to another network leaving 
Costs and Finance to the technical group (TG2). Mr Plank outlined the 
rationale for each indicator on the list and invited questions. The 
Chairman advised participants not to go to the fine detail of each 
indicator but to confine themselves to indicating areas needing 
further research. h spirited debate took place on the role that 
budgetary estimates as distinct from actual appropriations made should 
play and it was mentioned that the Secretariat was looking into this 
matter. Other points raised included the problems of distinguishing 
public from private sources of funds; the effect of using national 
accountancy criteria in presenting financial data for the education 
sector and the question how the boundaries of expenditure classified 
as education should be specified. 

The group then turned its attention to making detailed suggestions for 
the_ improvement of Table 7. Mr Plank complimented the Secretariat on 
the effort they had made to address the issues and expressed some 
concern about the way flows of funds especially from international 
organizations to countries and from one country to another were 
handled. Be would support a cautious move to improve existing 
questionnaires and praised the initiative taken by UNESCO towards the 
revision of ISCJtD. 



Overview of IEA work 

Mr Wiley gave an introduction on the work of ISA over the last years 
in neauurlng student outcomes. He specified the various policy 
decisions taken by that enquiry. He identified soate of the strengths 

ERIC 10 



- 9 - 



and weaknesses of th~ approach and indicated that IEA was moving to 
reduce the long dela> n getting out the results of studies* A lively 
discussion followed on ways in which poorer countries could be enabled 
to participate in IEA studies. 

Future UNESCO work on indicators 

Concerning the need for of a conceptual framework as a basis for 
future work, varied views were expressed, one of the key issues being 
the danger that an overt model, or even a formal category system, 
night freeze the development of the Indicator System and influence the 
flexibility of the World Education Report. One basis of the different 
perceptions related to the (extent to which it was desirable or 
feasible to create and use a category system which looked beyond 
present policy and associated programmes and targets. Those who 
advocated the identification and use of a formal category system felt 
that this was (a) feasible without detracting from its usefulness in 
serving current policy preoccupations and (b) desirable as experience 
showed that the priorities for policy changed over time and that a 
useful informational system a A its derived indicator systems, had to 
be robust enough to be still relevant in a new policy context. 

They pointed out also that if it were necessary to design and 
implement a new information system following each major policy shift, 
there would be long delays in putting such a system in place and 
getting sufficient time-series on the new basis; that almost all 
programs set up lo implement new policies would probably have been 
changed before they could be monitored in any overall systematic way. 
They pointed out that any systematic approach to identifying gaps as 
well as low priority items in an indicator system depended on the 
availability of a suitable category system. Those who took the view 
that information structures had to be closely tailored to policy 
priorities conceeded some of the above points, but preferred to have a 
statistical information system which operated within an incompletely 
formalized structure and in a sense responsive to intuitive 
perceptions of the need and priorities for change as policy changed. 

In asking He Hyland to introduce the discussion paper on this item, 
the Chairman pointed out that many of the topics had already been 
covered to a considerable extent in the general discussion. Be hoped 
that additional perspectives would emerge. Mr Hyland said it was 
desirable to envisage sustained dialogue between decision makers 
particularly those who critically influenced the allocation of money 
to producers of services and hence by implication to clients on the 
one hand and those responsible for the design and implementation of 
information on the other. It was essential especially at the 
international level, to engage in a vigourous dialogue between 
decision makers and information system designers. 

He said that decision makers used words and phrases which in the 
specific but iaplicit context in which the phrases warm used 
frequently Implied rather precise concepts and sometimes even a close 
range of operational options* The same phrases used in different 
specific contexts frequently referred to quite different concepts and 
operational options. Be referred to the various interpretations of 
the phrase "functional literacy" as an example of this situation. It 
is necessary therefore for thosa articulating long-term information 
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needs to be in close dialogue with information designers at critical 
decision points. The Division of Statistics in co-operation with the 
Education Sector of UNESCO had acted imaginatively in providing this 
opportunity for intimate dialogue. Given the opportunities provided, 
now was the tine to engage in a close and critical analysis of 
possibilities in conjunction with statisticians. Otherwise 
statisticians would have to make the necessary choices without the 
guidance which could ideally hav<; b<?en wade available at a critical 
tine. The current meeting had given soi e useful guidelines, and he 
suggested that the Division of Statistics intensify its efforts to 
stimulate critical and engaged dialogue on these momentous choices. 

The meeting was informed that the special questionnaire on basic 
education had been designed and distributed by the Division of 
Statistics as a response to the World Conference on Education for All. 
The data collected by this questionnaire would be available for the 
1993 World Education Report* The Division would welcome suggestions 
on priority items or on ways of analyzing or presenting the data. 
Looking to the possibility that this questionnaire might be issued 
again in the future and that some of the •successful" questions might 
be incorporated to the regular questionnaire, participants, who 
expressed appreciation at the emphasis given in the questionnaire to 
important qualitative dimensions, made a number of suggestion© for 
possible modifications. 

During the discussion on long-term options, one participant suggested 
that work should be initiated within UNESCO to operational ize the 
Jomtien document. This in itself could give a critical new focus to 
influence the design of information systems and related indicator 
development. Further work in the area of literacy was also essential. 
This would initially have to be experimental in character and at the 
research level in view of the difficulties already experienced in 
attempts to operational ize it. If successful, the results could 
possibly be used to measure key aspects of school outcomes. Reference 
to the existing IEA study was made and hopes were expressed that it 
could be helpful in future research effort. A similar approach would 
probably also have to be taken in the context of the importance of 
"measuring" learning achievements. This was particularly difficult 
when the results w*re used at the policy level. Any attempt to use 
results in this area to identify efficient suppliers of relevant 
services, would have to take background, and community factors into 
account and in effect develop a "measure" of a net concept of 
achievement, to be useful to clients or fair to suppliers. 

• The importance of training} activities, especially at regional level, 
was stressed. This had two tspects: m the identification and 
measurement of training activities in member countries and the 
possibility that UNESCO could form a perception of training needs, 
especially in the area of design and implementation of information 
systems and generate material and prospects. 

The suggestion in the discussion document that national publications 
be investigated systematically to Identify useful indicators nam 
widely endorsed as was the suggestion that countries be invited to 
publish key indicator a in one or more of a number of widely known 
languages* 
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Attention was drawn to the increasing emphasis on the importance of 
quality in all e<?jcation systems. While tne work done by IEA, IEPA 
and the other research organizations provided a good starting point, 
it was evident that further research wor.t would be required in some 
areas before a number of relevant issues could be covered in regular 
questionnaires. What could be done without further research should be 
evaluated for priority inclusion in such questionnaires* 

The need for data on language(s) and script (s) used in a variety of 
educational activities was stressed again* While this question 
affected some countries more than others, the critical importance of 
language (and script) for communication required special attention. 

One participant drew attention to the difficulty of getting data on 
conceptually simple but critically important issues and said the need 
for improvement in these areas should not be overlooked. He instanced 
such items as number of days attended as an example. More generally 
time spent and key aspects of curricula offeree and accepted t*ere 
equally important. 

Further information about the uses made of UNESCO and other indicators 
by decision makers, was required. The possibility that there were 
important and potentially damaging feed-back loops from inadequate or 
misinterpreted indicator systems, to resource allocation and other 
activities, needs to be investigated. Mr Uyland mentioned the concept 
of a client survey referred to in the discussion paper and he said 
that what he had in mind could be referred to as a "tagged" survey. 
Respondents could be tagged in terms of the "hats" they were wearing 
and this data would tie essential in evaluating replies. 

The question of evaluating and improving the need for accuracy in the 
data was revisited. The suggestion was made that it would be better 
to inform respondents not to provide data which were too inaccurate or 
too irrelevant and to notify UNESCO to this effect. Participants 
recognized that improving accuracy and relevance sometimes had a 
political dimension and almost always required additional resources; 
it was nevertheless essential to begin such improvements. 

References were made to various ongoing research projects in several 
regions and the hope was expressed that data from these projects as 
well as methodological and conceptual improvement associated with 
them, would be available in time to be taken into account in the 1993 
World Education Report. 

Mr Smyth expressed satisfaction from his point of view with the 
organization and the outcome of the meeting. He said, he believed that 
a meeting of a similar kind early next year in the context of the 1993 
World Education Report would be even more useful. It would be 
sufficiently ahead in time of the drafting of that report to exercise 
a great influence on it and on the data presented in it. 
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